Some changes occurred in src/tools/compiletest

cc @jieyouxu

Before: that's the only shown error when some annotation kind is missing.

Before: without missing annotation kinds (some stuff at the top is cropped, but the picture is clear).

After: with top and bottom messages merged, and suggestions added.

Nice! I'll play around with this tmrw on Monday.

☔ The latest upstream changes (presumably #140599) made this pull request unmergeable. Please resolve the merge conflicts.

@jieyouxu
By the way, do you think the line1:column1: line2:column2: part is useful in error messages?

I does two things

• it's displayed in annotation mismatch messages, but it mostly duplicates the file:line part which is also displayed.
  • the column part could be slightly useful in the past when we didn't have UI tests with captured .stderr, but probably not now
• it can actually be matched in line annotations, e.g. you can write //~ ERROR 28:16: cannot find value, but it seems to be an antipattern

Also, this part is not always added, it's only added to primary diagnostics, but not to labels or suggestions, for example.

I think maybe just drop it to reduce noise and duplication.
(The column can be moved to file:line:column instead of being fully dropped, if necessary.)

Another possible improvement is to display relative paths to test files instead of absolute paths, also to avoid noise, duplication and rightward shift.

The main requirement for the paths is to be "clickable", so you can quickly go from a file:line string in shell to that file and line in editor.

Relative paths here would work for me (vscode + rust is the only directory open in the workspace), but I'm not sure what other setups people can use.

Sorry for the wait, I played around with this locally, and I think it's an improvement. I have some feedback:

1. There's a starting message

   Blessing the stderr of in "/home/joe/repos/rust/tests/ui/compiletest-self-test/line-annotation-mismatches.stderr"

   where I think the actual source file path is missing. That might be pre-existing though.
   We could also maybe highlight the paths in yellow/magenta, but yeah.

2. I think we should only report the source file path once, /home/joe/repos/rust/tests/ui/compiletest-self-test/line-annotation-mismatches.rs, otherwise repeating that seems very noisy if we do it for every mismatch, and the rightwards drift is kinda annoying.

3. I don't find the line1:column1: line2:column2: useful in error messages themselves, especially not in error annotations like //~ ERROR 23:22 (unless someone wants to match on the exact column but...).

4. The column numbers might be useful if you want to jump in exactly, e.g. with neovim and whatever you can nvim path.rs:LINE:COL, or if there's > 1 diagnostics emitted on the same line? But for me, the line number is usually what matters. Agreed on stderr snapshots. I guess it might be useful if you don't check in a stderr snapshot (//@ dont-check-compiler-stderr)? I would be fine dropping it, and see if people actually have a specific use case where they want the exact column number too.

5. I'm fine with showing only a relative path, although if we go with (2), I don't think it would matter as much.

Misc design feedback

For the actual messages, I might even do sth like:

expected but with a different kind: tests/ui/compiletest-self-test/line-annotation-mismatches.rs:22:
          message: WARN: cannot find value `unresolved2` in this scope
    expected kind: ERROR
    reported kind: WARN

reported with a different message: tests/ui/compiletest-self-test/line-annotation-mismatches.rs:25:
    expected message: NOTE: not found in this scope
    reported message: 

If possible, we should report both the one we got and the one we expect (kind, message, line number).

Questions

• Also, how does this handle normalizations? E.g. //@ normalize-stderr (haven't played around with this much, I'll do that tmrw).

Let me know what u think.
@rustbot author

Reminder, once the PR becomes ready for a review, use @rustbot ready.